Fix testing perf telemetry: send measurements via measures arg#26028
Open
eleanorjboyd wants to merge 2 commits into
Open
Fix testing perf telemetry: send measurements via measures arg#26028eleanorjboyd wants to merge 2 commits into
eleanorjboyd wants to merge 2 commits into
Conversation
The discovery/run telemetry emitted the numeric performance fields (totalDurationMs, testCount, durationMs, requestedCount) in the properties bag instead of the measures argument of sendTelemetryEvent. These fields are annotated `isMeasurement: true`, and telemetry ingestion drops measurement-annotated fields that arrive as properties. As a result, across ~46M unittest.discovery.done events none carried a duration or test-count value, while the string categoricals (mode, trigger, failureCategory) came through fine. This meant there was no duration signal to evaluate the recent pytest discovery performance work (microsoft#25974, microsoft#25982). Move the numeric fields to the measures (2nd) argument at all five emit sites so they land in the Measures column, matching the established NATIVE_FINDER_PERF pattern. Also drop these fields from the event property type definitions (leaving the __GDPR__ annotations intact) and add notes so they are not reintroduced as properties. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
vritant24
approved these changes
Jul 2, 2026
benvillalobos
approved these changes
Jul 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The discovery/run testing telemetry added in #25980 emitted its numeric performance fields —
totalDurationMs,testCount(UNITTEST.DISCOVERY.DONE) anddurationMs,requestedCount(UNITTEST.RUN.DONE) — in the properties bag instead of the measures argument ofsendTelemetryEvent.These fields are annotated
isMeasurement: true, and telemetry ingestion drops measurement-annotated fields that arrive as properties. Verified against production telemetry: across ~46Munittest.discovery.doneevents in the last 14 days, zero carriedtotalDurationMsortestCount(in eitherPropertiesorMeasures), while the string categoricals (mode,trigger,failureCategory) came through fine. Themode/triggerfields survive because they're strings; the numeric ones silently vanish.This means there is currently no duration signal to evaluate the recent pytest discovery performance work (#25974, #25982) — the measurements were never actually recorded.
The correct pattern is confirmed by
NATIVE_FINDER_PERF, which passes its numbers via the measures (2nd) argument and lands them cleanly in theMeasurescolumn.Fix
Move the numeric measurement fields to the
measures(2nd) argument ofsendTelemetryEventat all five emit sites:resultResolver.ts—DISCOVERY.DONEsuccess path (totalDurationMs,testCount)controller.ts—DISCOVERY.DONEproject-error path (totalDurationMs) andRUN.DONElegacy path (durationMs,requestedCount)workspaceTestAdapter.ts—DISCOVERY.DONElegacy-error path (totalDurationMs)projectTestExecution.ts—RUN.DONEproject path (durationMs,requestedCount)Also removed these fields from the event property type definitions (the
__GDPR__annotations are unchanged and already correctly mark themisMeasurement: true) and added notes so they aren't reintroduced as properties.Verification
tsc: no new type errors from these changes.eslint+prettier: clean on all changed files.resolveDiscovery(5), projectRUN.DONEtelemetry (2), and workspace discovery (5).Impact
Once a build with this fix ships,
totalDurationMs/testCount/durationMs/requestedCountwill populate in theMeasurescolumn, enabling the before/after discovery-performance comparison (sliced bytestCountbucket ×mode). No user-facing behavior change.Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com